Medical imaging plays an important role in the screening, diagnosis, and/or treatment of many diseases because medical images enable a physician to view the internal anatomical structure of a patient or to visualize physiological or metabolic information. A variety of different imaging techniques or modalities can be used in clinical medicine. Some well known techniques/modalities include X-ray and computed tomography (CT), ultrasound, nuclear medicine, ultrasonic imaging, and magnetic resonance imaging (MRI). X-ray and CT, ultrasound, and MRI produce images of anatomical structure, whereas nuclear medicine produces images depicting metabolic uptake or biodistribution of radioactive compounds in various tissues (organs or tumors). Other modalities for imaging functional characteristics of physiological systems include functional MRI (fMRI), single photon emission computed tomography (SPECT), and positron emission tomography (PET). Still other modalities capture still images or video streams of internal structures by using a camera housed inside a scope. These modalities include colonoscopy, bronchoscopy, endoscopy, and capsule endoscopy.
Different techniques/modalities each have their strengths and weaknesses. For example, X-ray imaging has high spatial and intensity resolutions, shows bony anatomy with high detail, and is relatively inexpensive to use; however, it also presents the viewer with complex two-dimensional (2-D) views of superimposed anatomy. X-ray imaging can also have difficulty resolving soft tissue features.
MRI has the advantage of displaying three-dimensional (3-D) images of soft tissues with high contrast and high spatial resolution, and it does not involve ionizing radiation (as does X-ray and CT); however, MRI does not image bone well. CT imaging, based on X-ray absorption, produces 3-D images of bony anatomy, and increasingly, good definition of soft tissue, although MRI remains the preferred modality for viewing soft tissue.
Ultrasound imaging is easily portable, relatively inexpensive, and does not involve ionizing radiation. It has high spatial resolution and is extremely fast, enabling real-time frame capture rates. More recently, one unique and potentially powerful use of ultrasound has been discovered: measuring the elasticity of tissue, which can be useful in distinguishing tumor tissue from healthy tissue, for example, in the breast. A disadvantage of ultrasound is that it cannot easily image through gas or bones, making it difficult to obtain images of some organs.
Nuclear medicine provides images depicting metabolic information that can be early indicators of pathological conditions; however, it can be difficult to accurately pinpoint the location of anomalies in the body due to the lack of structural information in the images.
SPECT uses tomographic principles to provide a series of 2-D nuclear medicine images from nearby slices of tissue, effecting a 3-D nuclear medicine image; however, the spatial resolution can be slightly degraded. PET also is a tomographic technique that measures physiology and function, and provides images with higher spatial resolution and signal to noise ratio (SNR) than SPECT images. However, PET systems are very costly, because of the need for a cyclotron to produce positron-emitting nuclides. fMRI is not frequently used in clinical applications, with the exception of surgical planning, which aims to determine the areas of the brain that respond to specific cognitive tasks, in order to avoid those areas during surgery.
Scopes enable a visual inspection of the interior of a body lumen, such as the bronchi (bronchoscopy), the colon (colonoscopy), or upper gastrointestinal tract (endoscopy). Capsule endoscopy does not actually use a scope, but rather a swallowable capsule containing a camera that captures images while traveling through the entire gastrointestinal tract. Capsule endoscopy is more comfortable for the patient than endoscopy, and allows visualization deep within the intestines. However, the capsule and/or camera cannot be controlled or fixed on certain areas of interest, as can be done with a scope.
In some clinical applications, two or more modalities are used to capture medical images. In some applications, an image from one modality is used to screen for a disease, and then a subsequent image from another modality (usually of a higher resolution and/or diagnostic utility) is captured to verify a diagnosis or to gauge the progression of the disease. One example of this is when chest (X-ray) radiography is used to screen for lung nodules, lung cancer, or other respiratory diseases. Suspicious findings may cause a radiologist to order CT imagery to provide a high-resolution 3-D visualization of the affected area. Another example is (X-ray) mammography that is used to screen for breast cancer; a positive indication of breast cancer may then require a 3-D MRI of the breast for further investigation of tumors. A more recent example of a screening procedure is the use of CT imagery for virtual colonoscopy; the identification of suspicious regions or suspected polyps could result in a follow-up colonoscopy.
In addition to the use of images from multiple modalities in a screening/verification process, another common use of multi-modal medical imaging is to provide both anatomical and functional information. For example, in brain imaging, when cancer is suspected or diagnosed, CT and/or MR images may be captured to show the structure of the brain and any abnormalities, and PET or SPECT images may be captured to show any metabolic behavior of tumors or lesions. The combination of CT and PET imagery is used in the chest as well, in order to examine images of the lungs, liver, and kidneys. The CT/PET combination has been well received and devices capable of capturing both modalities simultaneously have been emerging in the marketplace (General Electric's Discovery LS PET/CT system and Siemens' biograph™ are two examples).
In other situations, images from multiple modalities may be captured even if the modalities provide some sort of structural information. CT and MR images of the brain or abdomen, or of orthopedic sites, may be captured at different times. The emergence of sonoelasticity, or measuring elastic properties of tissue using ultrasonic imaging, can be used to examine elasticity of the breast, providing structural information that complements mammograms or MR breast images.
However, even though medical imaging modalities provide a wide variety of visual information, many diseases are difficult to detect or diagnose. For example, one source estimates that around 20% of breast cancer cases are not detected by mammography.
In efforts to detect various types of cancers and of other diseases, many researchers have developed computer-assisted detection/diagnosis (CAD) techniques that aid the radiologist in detecting abnormalities.
CAD techniques directed to mammography are known. Refer, for example, to U.S. Pat. Nos. 5,633,948, 5,732,697, 5,941,832, 6,075,878, 6,266,435, 6,272,233, 6,418,237 and 6,553,356, and U.S. Patent Application Nos. 2001/0031076 and 2002/0057826.
CAD techniques applied to the detection of pulmonary nodules are also known. Refer, for example, to U.S. Pat. Nos. 5,539,838, 5,825,936, 5,881,124, 5,987,094, 6,125,194 and 6,609,021, U.S. Patent Applications Nos. 2003/0076992, 2003/0095696, 2003/0099388, and 200/30105395, and European Patent Nos. EP1,129,426, EP1,249,006, and EP1,395,165.
CAD applied to the detection of colorectal cancer is described in U.S. Pat. Nos. 4,981,783, 5,628,314.and 5,983,211, and U.S. Patent Application Nos. 2002/0187502, 2002/0022240, and 2003/0032860.
Osteoporosis and bone disease are the subject of CAD techniques in U.S. Pat. Nos. 4,913,157, 5,247,934, 5,673,298, 5,817,020, 5,902,240 and 6,143,506.
Even though these prior art techniques assist the medical practitioner in detecting and/or diagnosing disease, they apply to individual medical images or to multiple medical images from a single modality.
Some progress has been made in aligning and/or fusing images from multiple modalities. For example, U.S. Pat. No. 6,266,453 (Hibbard) is directed to a system for displaying multimodal image data (CT+MRI or CT+PET brain images) on a graphical user interface (GUI), enabling manual or automatic registration and fusion. U.S. Pat. No. 6,539,127 (Roche) relates to a method for registering general multimodal image data, based on correlation ratios between the data sets. U.S. Pat. No. 6,640,130 (Freeman) is directed to a system for fusing anatomic images with spectroscopic images of tissue or organs. U.S. Patent Application No. 2003/0216631 (Bloch) employs free form deformations to register PET and CT thoracic and abdominal images.
Although these methods and systems related to aligning and fusing images from multiple modalities, they do not address how to utilize the multimodal imagery to provide an enhanced method for detecting and/or diagnosing disease.
Existing CAD methods and systems apply to individual medical images or to multiple medical images from a single modality, and as such, they are not optimal when applied to a collection of images from multiple modalities. Registration and fusion techniques, while providing visually meaningful information to the medical practitioner, do not provide the same level of assistance as CAD systems.
The present invention addresses the problems and limitations of the prior art by providing a system and method for computer assisted detection and/or diagnosis of disease or abnormalities utilizing imagery from multiple modalities.